<p>
  As we know, a stock market index tracks the price changes of a select group of stocks and compiles those stock price changes into a single value.
  For example, S&amp;P500 is composed of only 500 large-cap stocks. A broad market index is characterized by including stocks from companies of all sizes(large, mid and small-cap based on their values).
  The most popular U.S. broad market indexes include the Russell 3000, the Wilshire 5000 Total Market Index and the MSCI U.S. Broad Market Index. Those broad-based market indexes attempt to cover the entire market
  and their return can be a good benchmark of the current market state.
</P>
<div class="section-example-container">
<pre class="python">
class MomentumandStateofMarkeFiltersAlgorithm(QCAlgorithm):
    def Initialize(self):
        self.SetStartDate(2011, 1, 1)
        self.SetEndDate(2018, 1, 1)
        self.SetCash(100000)
        # add Wilshire 5000 Total Market Index data from Dropbox
        self.AddData(Wilshire5000, "W5000", Resolution.Daily)
        # calculate the index yearly return
        self.W5000Return = self.ROC("W5000", 252)
        # initialize the RateOfChange indicator of Wilshire 5000 total market index
        history = self.History(["W5000"], 500, Resolution.Daily)
        for tuple in history.loc["W5000"].itertuples():
            self.W5000Return.Update(tuple.Index, tuple.value)
</pre>
</div>
<p>
  In this algorithm, we choose the Wilshire 5000 Total Market Index to be the market state measure. For the period of index return, longer horizons should capture more dramatic changes in the state of the market, but longer horizons also reduce the number of observations of changes in the market's state. Here we choose 12 months return according to the paper <a href="http://papers.ssrn.com/sol3/papers.cfm?abstract_id=299927">Market States and Momentum</a> from Guttierez, Cooper and Hameed.
  The daily index price comes from the <a href="https://finance.yahoo.com/quote/%5EW5000/history?p=%5EW5000">Yahoo Finance</a>.
</P>
<div class="section-example-container">
<pre class="python">
class Wilshire5000(PythonData):
    "Class to import Wilshire 5000 Total Market Index data from Dropbox"

    def GetSource(self, config, date, isLiveMode):
        return SubscriptionDataSource("https://www.dropbox.com/s/z9rof4fr9cqzgpt/W5000.csv?dl=1",
                                      SubscriptionTransportMedium.RemoteFile)
    def Reader(self, config, line, date, isLiveMode):
        if not (line.strip() and line[1].isdigit()): return None
        index = Wilshire5000()
        index.Symbol = config.Symbol
        try:
            # Example File Format: (Data starts from 01/04/2010)
            # Date    Open         High         Low          Close        Adj Close    Volume
            # 1/4/10  11549.13965  11749.37012  11549.13965  11743.54004  11743.54004  0
            data = line.split(',')
            index.Time = datetime.strptime(data[0], "%Y-%m-%d")
            index.Value = Decimal(data[5])
        except:
            return None
        return index
</pre>
</div>

<p>
  The investment universe contains all stocks on NYSE and NASDAQ with a price higher than $1. We use the momentum indicator to gauge the momentum effect.
  In <code>CoarseSelectionFunction</code>, the MOM indicator value for each symbol in coarse is updated with the adjusted price and saved in the dictionary <code>self.mom</code>.
</p>
<div class="section-example-container">
<pre class="python">
def CoarseSelectionFunction(self, coarse):
    coarse = [x for x in coarse if (x.HasFundamentalData and x.AdjustedPrice > 1)]
    for i in coarse:
        if i.Symbol not in self.mom:
            self.mom[i.Symbol] = SymbolData(i.Symbol, self.lookback)
        self.mom[i.Symbol].MOM.Update(self.Time, i.AdjustedPrice)
</pre>
</div>
<p>
  When the indicator is ready, stocks are then sorted based on the previous six months momentum value.
  Top 20 Stocks with the highest MOM are in the long stock list, 20 Stocks with the lowest MOM are in the short stock list.
</p>
<div class="section-example-container">
<pre class="python">
self.MOMReady = {symbol: SymbolData for symbol, SymbolData in self.mom.items() if SymbolData.MOM.IsReady}
if self.MOMReady:
    # sort stocks by 6 months' momentum
    sortByMOM = sorted(self.MOMReady, key = lambda x: self.MOMReady[x].MOM.Current.Value, reverse = True)
    self.long = sortByMOM[:20]
    self.short = sortByMOM[-20:]
    return self.long+self.short
</pre>
</div>

<p>
  The strategy is rebalanced monthly. At the beginning of each month, we identify the state of the market.
  If the market’s one-year return is positive, we define the state of the market as "UP" otherwise the state is "DOWN".
  When the market is in "UP" state, we go long on the previous six-month winners (highest Momentum) and goes short on the last six-month losers (lowest Momentum).
  Stocks are equally weighted. If the market is in "DOWN" state, we liquidate all asset holdings and invest in the long-term Treasury bond ETF to control the downside risk.
</P>
<div class="section-example-container">
<pre class="python">
def OnData(self, data):
    if self.month_start and self.selection:
        self.month_start = False
        self.selection = False
        if self.long is None or self.short is None: return
        # if the previous 12 months return on the broad equity market was positive
        if self.W5000Return.Current.Value > 0:
            stocks_invested = [x.Key for x in self.Portfolio if x.Value.Invested]
            for i in stocks_invested:
                if i not in self.long+self.short:
                    self.Liquidate(i)
            short_weight = 0.5/len(self.short)
            # goes short on the prior six-month losers (lowest decile)
            for short_symbol in self.short:
                self.SetHoldings(short_symbol, -short_weight)
            # goes long on the prior six-month winners (highest decile)
            long_weight = 0.5/len(self.long)
            for long_symbol in self.long:
                self.SetHoldings(long_symbol, long_weight)
        else:
            self.Liquidate()
            self.SetHoldings(self.tlt, 1)
</pre>
</div>
